Papers with NLP practitioners
100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask (P18-5)
Copied to clipboard
| Challenge: | This tutorial aims to provide information about semantics and pragmatics, as understood in linguistics, in a way that’s accessible to and useful for NLP practitioners with minimal (or even no) prior training in linguistics. |
| Approach: | This tutorial presents a selection of useful information about semantics and pragmatics, as understood in linguistics. |
| Outcome: | The aim of the tutorial is to present useful information about semantics and pragmatics, as understood in linguistics, in a way that’s accessible to and useful for NLP practitioners with minimal (or even no) prior training in linguistry. |
Integrating Ethics into the NLP Curriculum (2020.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to teach students how to ethically apply NLP . the tutorial will focus on examples for university classrooms, but ideas can extend to company-internal workshops or tutorials in a variety of organizations. |
| Approach: | a tutorial aims to empower NLP researchers and practitioners to teach others about ethical NLP . the tutorial will focus on examples for university classrooms, but ideas can extend to company-internal workshops . |
| Outcome: | a tutorial aims to teach students how to ethically apply NLP techniques . the tutorial will include examples for university classrooms, but ideas can extend to company-internal workshops . |
Generating Text from Language Models (2023.acl-tutorials)
Copied to clipboard
| Challenge: | a growing percentage of natural language processing tasks focus on the generation of text from probabilistic language models. |
| Approach: | They will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
| Outcome: | This tutorial will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
Endangered Languages meet Modern NLP (2020.coling-tutorials)
Copied to clipboard
| Challenge: | This tutorial will focus on NLP for endangered languages documentation and revitalization. |
| Approach: | This tutorial will focus on NLP for endangered languages documentation and revitalization . the goal is to motivate more NLP practitioners to work towards this important direction . |
| Outcome: | This tutorial will acquaint attendees with the process and the challenges of language documentation and revitalization. |
Attention Flows are Shapley Value Explanations (2021.acl-short)
Copied to clipboard
| Challenge: | Shapley Values are a popular type of explanation in machine learning, but leave-one-out and attention-based explanations still predominate in NLP. |
| Approach: | They propose to use attention flow to explain the importance of features, embeddings, and even neurons to explain credit assignment problems in cooperative game theory. |
| Outcome: | The proposed explanations can explain the importance of features, embeddings, and even neurons, but in NLP, leave-one-out and attention-based explanations still predominate. |
Transparent, Efficient, and Robust Word Embedding Access with WOMBAT (C18-2)
Copied to clipboard
| Challenge: | WOMBAT is a Python tool that supports accessing word embeddings from code. |
| Approach: | WOMBAT is a Python tool which supports NLP practitioners in accessing word embeddings from code. |
| Outcome: | WOMBAT is a Python tool which supports accessing word embeddings from code. |
FastFit: Fast and Effective Few-Shot Text Classification with a Multitude of Classes (2024.naacl-demo)
Copied to clipboard
| Challenge: | Few-shot prompting of large language models (LLMs) via API calls presents a unique challenge when dealing with a multitude of classes that share similar semantic meanings. |
| Approach: | They present a Python package that integrates batch contrastive learning and token-level similarity score to provide fast few-shot classification. |
| Outcome: | The proposed method significantly improves multi-class classification speed and accuracy across English and Multilingual datasets. |
Leveraging Pre-trained Checkpoints for Sequence Generation Tasks (2020.tacl-1)
Copied to clipboard
| Challenge: | Unsupervised pre-training of large neural models has revolutionized Natural Language Processing. |
| Approach: | They propose to use pre-trained checkpoints for Sequence Generation to initialize a Transformer-based sequence-to-sequence model that is compatible with these checkpoint. |
| Outcome: | The proposed model is compatible with pre-trained BERT, GPT-2, and RoBERTa checkpoints and achieves state-of-the-art results on Machine Translation, Text Summarization, Sentence Splitting, and Sentance Fusion. |
Label Noise in Context (2020.acl-demos)
Copied to clipboard
| Challenge: | Label noise—incorrectly or ambiguously labeled training examples—can negatively impact model performance. |
| Approach: | They propose a noise-detection method that uses an example's neighborhood within the training set to reduce false positives and provide an explanation as to why the ex ample was flagged as noise. |
| Outcome: | The proposed method outperforms the state-of-the-art on precision and F0.5-score on short-text classification datasets. |
Variationist: Exploring Multifaceted Variation and Bias in Written Language Data (2024.acl-demos)
Copied to clipboard
| Challenge: | Existing tools that inspect and visualize language data are limited in their capabilities. |
| Approach: | They propose a highly-modular, extensible, and task-agnostic tool that inspects language variation and bias across multiple variables, language units, and diverse metrics. |
| Outcome: | The proposed tool can inspect and visualize language variation and bias across variables, language units, and diverse metrics that go beyond descriptive statistics. |
How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee Language (2022.acl-long)
Copied to clipboard
| Challenge: | There are an estimated 6000 to 7000 spoken languages in the world, and at least 43% of them are endangered. |
| Approach: | They propose three principles that may help NLP practitioners foster mutual understanding and collaboration with language communities and three ways in which NLP can potentially assist in language education. |
| Outcome: | The proposed methods can be used to enrich Cherokee language resources with machine-in-the-loop processing and to provide language education. |
Measuring Intersectional Biases in Historical Documents (2023.findings-acl)
Copied to clipboard
Nadav Borenstein, Karolina Stanczak, Thea Rolskov, Natacha Klein Käfer, Natália da Silva Perez, Isabelle Augenstein
| Challenge: | digitised historical documents suffer from errors introduced by optical character recognition (OCR) and are written in an archaic language. |
| Approach: | They investigate the continuities and transformations of bias in Caribbean historical newspapers during the colonial era . they use distributional semantics models and word embeddings to measure gender, race, and intersectional biases. |
| Outcome: | The authors show that gender and racial biases are interdependent and their intersection triggers distinct effects. |
Can Explanations Be Useful for Calibrating Black Box Models? (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models are often used as black boxes to adapt to new domains, but there is no single recipe for making them work. |
| Approach: | They propose to use black box models to improve their performance on new domains by leveraging explanations of their behavior. |
| Outcome: | The proposed method improves model generalization performance on two tasks using explanations. |
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList (2020.acl-main)
Copied to clipboard
| Challenge: | In a recent study, we show that holding-out data can overestimate performance of NLP models. |
| Approach: | They propose a task-agnostic methodology for testing NLP models using a matrix of general linguistic capabilities and test types that facilitate comprehensive test ideation. |
| Outcome: | The proposed method identifies critical failures in commercial and state-of-the-art models. |
Scaling Laws for BERT in Low-Resource Settings (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models require huge training corpora, which is unobtainable for most NLP practitioners. |
| Approach: | They propose power-law formulas that relate model size, corpora size and computation power to find the optimal settings in advance given a fixed budget. |
| Outcome: | The proposed models perform better on MLM and NLU tasks on four languages of different linguistic characteristics. |
NYTWIT: A Dataset of Novel Words in the New York Times (2020.coling-main)
Copied to clipboard
| Challenge: | Novel words, or Out-Of-Vocabulary words, are a pervasive problem in modern natural language processing. |
| Approach: | They present a dataset of over 2,500 novel English words published in the New York Times . they use uncontextual and contextual predictions to predict novelty class . |
| Outcome: | The proposed dataset includes over 2,500 novel English words published in the New York Times between November 2017 and March 2019 . baseline results show that there is room for improvement even for state-of-the-art NLP systems . |
Model-tuning Via Prompts Makes NLP Models Adversarially Robust (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained models are typically adapted to downstream tasks by appending a randomly initialized multilayer perceptron to their topmost representation layer and fine-tuning the entire model on a downstream task. |
| Approach: | They propose to append a multilayer perceptron to a CLS token and fine-tune the entire model on a downstream task. |
| Outcome: | The proposed model-tuning via prompts outperforms adversarial training-based state-of-art defenses by 3.5% and improves against adversarials by 8% over standard methods. |
We Need to Talk About Reproducibility in NLP Model Comparison (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that standard splits produce low reproducible and unreliable conclusions . reproducibility of empirical experimental conclusions is a problem in NLP domain . |
| Approach: | They propose to transform the reproducibility of a model comparison into a probabilistic function . they propose to use a regularized corpus splitting strategy to estimate the model's performance . |
| Outcome: | The proposed estimator achieves a high SNR and significantly increases reproducibility. |
The Only Way is Ethics: A Guide to Ethical Research with Large Language Models (2025.coling-main)
Copied to clipboard
Eddie L. Ungless, Nikolas Vitsakis, Zeerak Talat, James Garforth, Bjorn Ross, Arno Onken, Atoosa Kasirzadeh, Alexandra Birch
| Challenge: | Existing literature on the ethical aspects of large language models (LLMs) is lacking a single practical guide on the subject. |
| Approach: | They propose to translate ethics literature into concrete recommendations for computer scientists by presenting an open and living resource for NLP practitioners and those tasked with evaluating the ethical implications of others’ work. |
| Outcome: | The proposed guide is an open and living resource for NLP practitioners and those tasked with evaluating the ethical implications of others’ work. |
Specifying Genericity through Inclusiveness and Abstractness Continuous Scales (2024.lrec-main)
Copied to clipboard
| Challenge: | Using a pilot study, we created a small but crucial annotated dataset of 324 sentences, demonstrating the framework’s effectiveness in capturing nuanced aspects of genericity. |
| Approach: | They propose a framework for fine-grained modeling of noun phrases' genericity in natural language using a small but crucial annotated dataset of 324 sentences. |
| Outcome: | The proposed framework can be used to model genericity of noun phrases in natural language and can be easily compared with existing binary annotations. |